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I. INTRODUCTION 


Trust bonds relationships, both personal as well as professional. Within an 
organization, trust facilitates sharing of information, improved morale, flexibility, and 
open communication. It can enhance efficiency by reducing the need for governance, a 


time consuming effort for managers and supervisors (Van de Ven, 2004). 


Fukuyama (1995) states the United States was one of the first countries to develop 
large, modern professionally managed corporations. This innovative creation of larger 
organizations was possible because the American culture allowed organizations to build 
networks beyond family ties, relying instead on the socially bred high degree of trust 


among fellow citizens. 


The trust that helped establish larger organizations has been declining over the 
past five decades according to Bruhn (2001). He reviewed numerous polls and studies on 
trust, honesty, integrity, and commitment, and concluded, “Today, nearly two in three 
Americans believe that most people can’t be trusted; half say most people would cheat 
each other if they had a chance, and half say that most people are looking out for 


themselves (Bruhn, 2001).” 


Just as trust in U.S. society is changing, there are also new developments in 
organizations. A diversity of organizational structures continues to emerge and change 
and managers are no longer limited to the traditional hierarchical organization. 
Technological advances, globalization, competition, and workforce dynamics have driven 


these emerging and changing organizational structures. 


Organizational performance is impacted by the combination of these two 
changing factors, trust and structure. This project analyzes the effects of varying trust 


conditions and organizational structure on performance. 


The remainder of this section discusses differences in organizational structure and 


trust, and concludes with a broad description of this project’s research goal. 


A. ORGANIZATIONAL STRUCTURES 


Two primary structures are formal Hierarchy and flat, organic structures, also 
known as Edge structures (Alberts & Hayes, 2003). Bolman and Deal (2003) suggest 
that organizations are arranged to minimize problems while maximizing performance. 
They offer assumptions underlying structural design. First, organizations exist to achieve 
goals and objectives. Next, they seek efficiency and enhanced performance through 
specialization and division of labor. Lastly, the optimal form of coordination ensures that 


efforts of the organization’s members and units mesh properly. 


Mintzberg (1979) describes the basic blueprint of an organization through by 
depicting five different components: the strategic apex, the technostructure, the support 
staff, the middle line, and the operating core. Organizational structures differ from one 
another through different distributions and emphasis of each of the components. An 
introduction to the hierarchy is described in Mintzberg’s models of the simple structure 


and the machine bureaucracy. 


The simple structure consists of only two components, the strategic apex and the 
operating core. The leadership at the strategic apex provides decisions and information, 
which flow downward, to the workforce in the operating core. The top tier provides 
monitoring and direct supervision. Members within the core communicate upward with 
the leadership rather than with one another. This arrangement is often used in an 
environment involving simple and routine tasks, and a single person directs the work of 
the organization (Mintzberg, 1979). Startup companies often begin with a simple 
structure; an example is the business operated in a garage started by William Hewlett and 


David Packard (Bolman & Deal, 2003). 


The machine bureaucracy is comprised of many layers between the strategic apex 
and the operating core. The added layers of the machine bureaucracy are found in the 
middle line where management operationalizes the decisions from the apex, distributes 
information, and provides supervision to members of the operating core. The structure 
also incorporates large support staffs and technostructure. Like the simple structure, the 


machine bureaucracy is typically used in organizations where tasks are routine. Unlike 
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the simple structure, the machine bureaucracy’s top tier is concerned with larger strategic 
objectives than supervision (Mintzberg, 1979). McDonald’s organization incorporates an 


example of this structure (Bolman & Deal, 2003). 


The formal Hierarchy seeks to achieve control and stability, characterized with 
clear lines of authority and a “chain of command” (Ancona, Kochan, Scully, Van Maanen 
& Westney, 1999). This traditional structure has contended with new organizational 
forms over the past 20 years because of changing and dissolving organizational 
boundaries as a result of technological advances (Wigand, 2008), pressures of 
globalization, competition, and workforce dynamics (Bolman & Deal, 2003). The new 
forms emphasize a flattening of organizational structure, enhancement of networked 
capabilities, achievement of changing objectives through flexibility of operations, and 


reliance on diverse skills (Ancona et al., 1999). 


A new structure that has emerged is the Edge. This organizational design 
leverages networking by moving knowledge and power among geographically distributed 
participants to the edges of organizations (Leweling & Nissen, 2007). The Edge seems 
appropriate for the military because of its broadening involvement in a variety of military 
operations beyond combat. Examples include humanitarian assistance, homeland 
defense, and noncombatant evacuations (U.S. Department of Defense, 2006). These 
expanding mission sets require organizations to operate beyond routine tasks and 


transition to capabilities requiring flexibility. 


B. COALITIONS 


In the context of increasing technologies and global interaction, organizations 
increase in complexity, and therefore, collaborate with other organizations to accomplish 
goals requiring numerous functions and specialties. Task forces and teams assemble to 
harness the various capabilities of different organizations (Bolman & Deal, 2003). The 
military uses the concept of assembling organizations as evidenced by its involvement in 


numerous multinational coalitions (U.S. Department of Defense, 2000), interagencies 


(U.S. Department of Defense, 2006), and interorganizational networks (Kennedy, 2004; 
Erckenbrack & Scholer, 2004). An example of U.S. Department of Defense participation 
in a multinational coalition is the Multi National Force-Iraq, MNF-I (MNF-I, 2008). 


The MNF-I mission is to work in partnership with the Iraqi Government to 
contribute to the maintenance of security and stability of the country. Twenty-four 
countries participate in the coalition. The Multi National Core-Iraq, MNC-I, an 
organization within the coalition, is the tactical unit responsible for command and control 
operations in Iraq. The organization is geographically distributed in six regions 
throughout the country. Each region maintains a multinational force and an associated 


headquarters (MNF-I, 2008). 


Both the Edge and Hierarchy organizations are incorporated in the MNF-I 
coalition (MNF-I, 2008). The two forms are illustrated through differing forms of 
Australian participation in the coalition. An example of the Edge organization is the 
Coalition Counter Improvised Explosive Device Task Force. A small number of 
Australian Defense Force personnel work alongside members from other nations’ 
military forces to coordinate intelligence collection and Iraqi Forces training in support of 
the counter IED efforts (Australian Government, Department of Defense, 2008). The 
Australians also maintain a Hierarchy within the coalition. The HMAS Parramata is an 
Australian frigate, which patrols the Northern Persian Gulf to assist in the protection of 
Iraq’s offshore assets such as oil platforms (Australian Government, Department of 
Defense, 2008). The Australian crew aboard the Parramata is organized in the traditional 
hierarchy organization whereby the Commanding Officer maintains authority at the 
strategic apex and makes the decisions about the ship’s operations. He distributes 
information downward through the department heads in the middle line of the 


organizational structure. 
C. TRUST WITHIN COALITIONS 


Trust is a critical element in achieving a coalition’s goals and objectives. Trust is 
required because of interdependence and diversity (Mayer, Davis, & Schoorman, 2005). 


The increasing complexities of organizations results in interdependence on other 
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organizations, and therefore, vulnerabilities arise due to reliance on the other participants. 
Organizations within coalitions must therefore trust that other organizations will perform 
actions important to them (Mayer et al., 1995). Coalitions are composed of numerous 
participants and units resulting in increased diversity. Members of the coalition cannot 
rely on interpersonal or interorganizational similarities and common background and 
experience to enhance the willingness to work together (Berscheid & Walster, 1978; 
Newcomb, 1956), and therefore, must develop trust as a mechanism to enable them to 
work together effectively. Efficiency, performance, and coordination hinge on trust to 
ensure the elements of interdependence and diversity enhance a coalition’s operations, 


rather than hinder it. 


D. PURPOSE OF TRUST RESEARCH WITHIN THE CONTEXT OF 
DIFFERENT ORGANIZATIONS 


Military organizations have adapted over many centuries (Winnefeld & Johnson, 
1993; Allard, 1996; Palmer, 2005; Leighton, 1952). Military hierarchy organizations 
have enjoyed refinement through combat, training, and doctrinal development (Janowitz, 
1959). In contrast, applications of the Edge structure within military organizations have 
not been tested due to the recent introduction of the structure (Nissen, 2005). Increasing 
levels of interorganizational coordination within the military indicate that Edge 
organizations may be more suitable than Hierarchy command and control structures in 


certain circumstances (Leweling & Nissen, 2007; Powley, Fry, Barrett, & Bright, 2004). 


The opportunity exists to add to a body of knowledge concerning the comparative 
advantages for Edge organizations using laboratory experimentation, structured controls, 
and data collection and analysis. This research pertains to the efficacy of the Edge 
organization within the context of trust and mistrust conditions, set against the 


effectiveness of a hierarchy organization to provide a comparative analysis. 


Literature focused on organizational structures provided the groundwork upon 
which the research hypotheses were based. The research design is detailed and key 
findings and results offered. This research project closes with conclusions and 
recommendations for future experimentation to build upon research concerning 


organizational design and trust conditions. 


The following section is a literature review and discusses existing and current 
theories, methodologies, implementing trust and its influence on trust conditions in 
organizations. The emphasis is on trust factors affecting performance on Hierarchical 


and Edge organization types. 


I. LITERATURE REVIEW 


This section provides an overview of existing as well as current theories, 
methodologies, and implementation of trust (i.e., integrity, ability, and benevolence) and 
how it influences conditions of trust in organizations. It also focuses on how the factors 
of trust affect the performance in Hierarchal Organization types as well as Edge 
Organization types with the specific task of solving complex problems with a high rate of 
success. Even though the factors included in the section are separate and individually 
relevant in establishing trust in small group dynamics, each factor works together to form 
the system necessary for success in small organizations. This section concludes by the 
integration of trust, and its effects on military style command and control (C2) situations 
as well as autonomous Edge Organization types. It will also include the positive and 


negative results of Hierarchical Organization and Edge Organization types. 


A. RELEVANCE OF TRUST LINKED TO PERFORMANCE 


The role of trust in groups whose goal is to achieve a high level of success while 
working in groups has been extensively studied. Most of the studies were performed in a 
Hierarchical Organization type environment similar to those used in military settings. 
One central figure had overall control of the organization with a series of subordinate 
entities with specific tasks within the system. All control decisions were executed by the 


central control figure. 


Trust in the top is a significant determinant in how the subordinate group 
performs its tasks and how successful the results of these groups ultimately impacted. 
The following factors determine what contributes to trust or mistrust: information, 


influence and control. 


An authority figure that does not trust subordinates disseminates information 
significantly differently than an authority figure that does. Someone who does not trust 
other members of the team/unit will not be completely forthcoming with all relevant 


information needed for the individual to minimize exposure to the rest of the group 
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(Zand, 1972). The non-trusting member does not tend to want relevant contributions 
from other members of the group to exert their influence further. Although the non- 
trusting member exerts influence in terms of the proposal of goals, suggestions for 
reaching goals and metrics for evaluating success, two-way communications from other 
members of the team are highly discouraged. Yet, the non-trusting member expects 


subordinates to accept inputs readily (Zand, 1972). 


“Finally, one who does not trust will try to minimize his dependence on others. 
He will feel he cannot rely on other members of the group to abide by agreements and 
will try to impose controls on their behavior when coordination is necessary to attain 
common goals, but will resist and be alarmed at their attempts to control his behavior” 
(Zand, 1972, p. 230). When low-trust is established within a group, all the necessary 
means of communication such as dependence on other members of the group, 
coordination of information, adequate feedback from all associated members, and the 
need to attain a common goal ultimately leads to decreased ability to solve problems 


efficiently. 


Groups comprised of individuals with adequate levels of trust among its members 
act differently and solve problems more effectively than groups with low levels of trust. 
Groups that display high levels of trust are quantitatively better able to solve problems 
within the group because the members are more willing to be vulnerable when it comes 
to problem-solving efforts. Barriers to information dissemination, accuracy, relevance, 
and timeliness are not prevalent when groups have higher levels of trust since 
interdependence and reception to the influence of others is encouraged for timely 
accurate problem solving. “They will also accept interdependence because of confidence 
that others will control their behavior in accordance with agreements, (see Figure 1) and 
therefore will have less need to impose controls on others” (Zand, 1972, p. 231). Groups 
with high levels of trust tend to be more creative problem solvers with long-range 


relevant solutions compared with their low-trust counterparts. 


Trust 


Increase one's wukvrabiliy” 
to others whose behavior one 


Cortrol Peformation 


Dkcbhs more acowrate relevant 
Acceptmore tterdependence vith and complete data about the : 
others. beupose less procedure to probkm, one s thoughts and one's 
cortrol others. Greater confidence. feelmgs 
Greater confidence others will do 
vrhat they agree to. Gre ater 
Comminent to do what one agreed 
to. 










Acceptmore if buence from 
others in selection of goak, 
choice of methods, evahuation of 
progress. 


Figure 1. | A Model of the Relationship of Trust to Information, Influence and 
Control, (From: Zand, 1972). 


Establishment of trust is a significant and relevant element in Edge Organization 
types as well as Hierarchy organization types. Ability, benevolence, and integrity are 
equally significant tangible aspects that help build trust and promote its use throughout 
the life of the organization type of choice, be it Edge or Hierarchy. Ability, benevolence, 
and integrity are individually beneficial when attempting to establish trust in a relatively 
short period of time; however, when ability, benevolence, and integrity are combined 
holistically, each personality trait adds value and validity to the individual or individuals 


comprising the organization when establishing trust is paramount. 


B. ABILITY 


The domain of the ability is specific because the trustee may be highly competent 
in some technical area, affording that person trust on tasks related to a specific area. 


However, the trustee may have little aptitude, training, or experience in another area, for 
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instance, in interpersonal communication. Although such an individual may be trusted to 
perform analytic tasks related to the technical area, the individual may not be trusted to 


initiate contact with an important customer (Mayer, Davis, & Schoorman, 1995). 


The use of ability in the study of Hierarchical Organization types can be difficult 
to quantify since ability is a quality that tends to be very specific and not always able to 
be used outside of the member’s specialty or area of expertise. As discussed earlier, 
ability, benevolence, and integrity are units of measure or defining factors of group 
behavior that as a system lead to rapid development of trust in small groups to effectively 
solve problems. Ability is a significant measure in the system of developing trust; 
however, ability tends to be a very specific aspect in establishing competence. Even 
though an individual may have a high level of ability in one area such as quantum 
physics, the same individual may not have any aptitude, training, or experience in a field 
such as the social dynamics that determine the selection of targets by terrorist 
organizations. For instance, a terrorist of Hispanic descent may pick a different target to 
destroy than a terrorist of Arab descent based on the culture in which the terrorist was 
raised. The social dynamics of Latin American cultures may dictate that a military target 
is more significant to western societies and thus choose it as a viable target that focuses 
on western centers of gravity. On the other hand, a terrorist of Arabic descent may 
determine that targets of economic importance may be more significant to western 
societies. Although such an individual may be trusted to do one job in an area of 
expertise, trust to perform tasks in areas outside the area of expertise may not be granted. 


Thus, trust is domain specific (Zand, 1972). 


C. BENEVOLENCE 


Benevolence is a trait that is very specific to a situation or job being performed, 
and thus, changes with each individual situation. Benevolence in a military setting may 
be viewed completely differently than benevolence in a civilian business setting. 
Benevolence is also specific to individuals. Another individual in the same organization 
type may see what one person deems to be benevolent behavior as not benevolent but 


instead the behavior could be viewed as neutral or even malevolent. Benevolence 
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focuses on the relationship of the trustor and the trustee, how the relationship affects 
performance, and explains the potential long-term positive effects that benevolence can 
have in a relationship where gaining and maintaining trust is paramount. Benevolence 
suggests that the trustee has some specific attachment to the trustor (Mayer, Davis, & 


Schoorman, 1995). 


An example of this relationship can be found in many interactions in the military 
as well as civilian organizations. Mentoring or apprenticeships are relationships that 
require a sense of benevolence on the behalf of the trustee. There is no benefit for the 
trustee being helpful to the trustor; however, if the trustee is benevolent, the desire is to 
see the trustor succeed with no reward or benefit on the part of the trustee. “Benevolence 
is the perception of a positive orientation of the trustee toward the trustor” (Mayer, Davis, 
& Schoorman, 1995, p. 718). Benevolence in work group relationships can be a 
comerstone of trust in a working relationship of this type because the trustor perceives 
that the trustee is helping out of no benefit to himself, but of a genuine want or need to 


help the new individual become a contributing member of the group or team. 


D. INTEGRITY 


Integrity is difficult to measure because perceptions can change from place to 
place or organization to organization. For instance, subordinates, peers and superiors 
perceive military officers in the environment of a military organization to have inherently 
high levels of integrity. This integrity is based on naval traditions of behavior, personal 
characteristics, and honesty of all naval officers. However, in a different organization, 
the standards that dictate a military officer’s integrity may be seen as a negative in a 
different organization. Therefore, the perceived integrity of the same individual could be 
greatly diminished dependent on the organizational circumstances. A naval officer with 
high levels of integrity may not be seen in the same vein when applying for a 


management position at Enron. 


In conclusion, there are several factors to consider when attempting to establish 
trust with the mission of correctly and efficiently solving complex problems. Without 
properly establishing trust (i.e., ability, benevolence, and integrity) the ability of the 
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group to perform tasks could be greatly diminished, leading to the unsuccessful 
completion of the objective. Although Hierarchy Organization and Edge Organization 
types are dissimilar in how they are managed, it is believed that high levels of trust, 


(based on these dimensions) will lead to successful problem solving in both organization 


types. 


E. HYPOTHESES 


In this project, considering trust and its effect on edge organization types as well 
as effects on hierarchy organization types, we broke out the two organizations based on 
two measurable trust conditions. Our first condition was the organization type; which are 
edge and hierarchy, and whether the trust condition affects solution outcomes. Second, 
we measured the trust conditions and their affects on solution outcomes. The first set of 
hypotheses suggests a flat, organic organizational form will perform better than a 
hierarchical form. 

The hypotheses were as follows: 

e Hypothesis 1: The Edge organization type will outperform Hierarchy 

organization type regardless of trust condition. (Main Effect) 


e Hypothesis 1(a): The Edge trust condition will outperform Hierarchy 
trust condition. (Interaction) 


e Hypothesis 1(b): The Edge mistrust condition will outperform Hierarchy 
mistrust condition. (Interaction) 
In the second set of hypotheses, we suggest that performance in the trust (over the 
mistrust) condition will be higher. 


e Hypothesis 2: Trust will outperform mistrust regardless of organization 
type. (Main Effect) 


e Hypothesis 2(a): Trust in the Edge organization type will outperform 
mistrust in Edge organization type. (Interaction) 


e Hypothesis 2(b): Trust in the Hierarchy organization type will outperform 
mistrust in Hierarchy organization type. (Interaction) 
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The following section discusses the methodology used in this project. It also 
mentions the subjects and the groups used. The different communication protocols 


employed are elaboration upon as well. 
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Hl. METHODOLOGY 


This section discusses the methodology utilized in this project. In this section, we 
describe the research design and the structure of the experiments. Building directly upon 
research by Leweling and Nissen, and drawing on the literature of Zand, and Mayer 
Davis and Schoorman, we use the ELICIT multiplayer intelligence game to test subjects’ 
ability to solve problems under combined treatments of trust and organizational structure. 
This section begins with a description of the ELICIT environment followed by a detailed 


review of the subjects, groups, and protocols used in the experiments. 
A. ELICIT ENVIRONMENT 


This experimental design uses the ELICIT multiplayer intelligence game to place 
the subjects in a collaborative problem solving environment. Subjects log in and play the 
game via client applications on separate, networked computer workstations. ELICIT 
provides subjects with the challenge of identifying all of the key elements of a terrorist 
plot in an attempt to prevent the attack from occurring. Its design is similar to the Parker 
Brothers’ board game “Clue” in that it requires each player to analyze clues and combine 


assessments with other players to answer the following questions. 
Who: Who planned/led the effort to attack? Who will execute the attack? 
Where: In what country will the attack take place? 
What: What is the specific target or event to be attacked? 
When: The three factors of “when” are month, day and time of the attack. 


Each of the 68 clues called “factoids” describe a finite detail of the terrorist plot, 
but each factoid is incomplete in addressing only one aspect (who, what, where, when), 
and not every factoid is necessary or helpful in assembling an accurate and complete 
answer. The factoids are distributed to the 17 players as follows: two factoids are released 
to each player as game play begins and another after an additional five minutes of game 


play. At twenty minutes, all factoids have been distributed, and all necessary facts 
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regarding the terrorist plot are available for solution. No single player can solve the 
problem with the given factoids set so the players must collaborate to solve the plot and 


win the game. 


The subjects interact with ELICIT via a website based display which shows 
various user status frames. The list screen displays all factoids that a particular player has 
received. After the initial factoid distribution, a player’s List screen would display the 
two factoids distributed by the server, for example, “The Lion is involved,” and, “The 
Violet Group only operates in Daylight.” These functions are associated with four, 
separate windowed screens, each corresponding to the pertinent questions (i.e., who, 
what, where, when) regarding the terrorist plot. One tabbed screen exists for viewing and 
accumulating each category of factoid and includes information regarding who, what, 
where and when data. During game play, each subject has access to a set of five 
functional capabilities supported by the network client that allows the user to take one of 


five actions with the accumulated factoids: 1) List, 2) Post, 3) Pull, 4) Share, and 5) 


Identify. 

° List creates a list of chosen factoids selected by the player viewable in the 
main game window. 

° Post enables a player to have one or more factoids displayed on a common 
screen that can be viewed by all other players simultaneously. This 
represents one of two mechanisms for sharing information in the game 
(e.g., verbal communication is prohibited in this experiment protocol). 

e Pull represents the complement to Post. Any player can add posted 
factoids to the List pane. 

° Share represents the second mechanism for sharing information in the 
game, and enables players to send factoids only and directly to one 
another. 

@ Identify represents the manner in which subjects publish their “solutions” 


to the problem, once they have adequate factoids to propose a complete 
answer (i.e., who, what, where, when) regarding the terrorist plot. 


Multiple scenarios of game play are possible in ELICIT. Each session is 
structurally similar but contains a different answer set. Each scenario set includes 17 
players (and pseudonyms) and a set of 68 factoids. However, the factoids and the 
sequence of distribution are unique to each version. The potential exists to play the game 
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multiple times, even with the same group of subjects. After the game is complete, the 
moderator shuts down the server application, and exportable log files are created for 
extraction and analysis. The log files consist of time-stamped entries for every data 
transaction in the game including game start, players’ names (pseudonyms), details of 
factoid distribution (what specific factoid and to whom), all interactions between players 


including Post, Pull, Share, Identify, and game end. 


B. SUBJECTS 


Subjects for this study are 135 first-quarter graduate students at the Naval 
Postgraduate School (NPS) enrolled in a core organizational behavior course. Students in 
the Graduate School of Business and Public Policy (GSBPP) consist of active duty 
members of the United States Army, Air Force Navy and Marine Corps as well as foreign 
students from the School of International Graduate Studies (SIGS). A total of 135 
subjects possess between one and 18 years of work experience. All subjects have 
undergraduate college degrees as well as direct military service. This representative 


sample serves to enhance the external validity of the study. 


Despite the considerable level of realism designed into the ELICIT game, the 
information-sharing and processing task is limited in the scope of communication and 
interpersonal interaction. It is also structured so that subjects can be evaluated in as little 
as 45 minutes of game play. The networked-computer, ELICIT-mediated task 
environment does not enable all of the typical communication modalities such as 
telephone, video teleconference, face-to-face communication, email and group 
interaction. This structure serves to enhance internal validity through control but limits 


the external validity of the study as applied to more typical group communication. 


oe GROUPS 


Subjects were randomly assigned to one of eight groups of 17 members each. A 
list of subjects was ranked in decreasing order of seniority to ensure an equal distribution 
of age and experience among the eight groups. Each group contained an equally 


distributed representation of military service branch, organizations, officer subspecialties, 
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genders, and country of service to accurately create the conditions present in Joint and 
Coalition military organizations. Subjects were assigned to one of the eight work groups 


as follows. 


Test day one Test day two 

A: Hierarchy, Trust A: Hierarchy, Trust 

B: Edge, Trust B: Edge, Trust 

C Hierarchy, Mistrust C: Hierarchy, Mistrust 


D: Edge, Mistrust D: Edge, Mistrust 


Figure 2. Treatment Matrix 


The most senior subjects were assigned to the role of “Alex,” the leader in the 
ELICIT game. This is similar to the manner in which professional analysts are assigned 
to specific roles in operational intelligence organizations in the field, and hence helps to 
ground this experiment though conformance to practice. This approach contrasts a little 
with that of randomized assignment imposed in some prior studies, emphasizing a 


concern for realism over replication. 
D. PROTOCOLS 


Subjects reported to a network classroom on their assigned day for the 
experiment. Subjects were assigned to play specific roles (e.g., as identified via 
pseudonyms) in the game. Subjects were briefed about the experiment in their 
organizational behavioral class and formally consented to participate in it. When all 
students were seated and the ELICIT clients connected with the server, subjects sat down 
at the appropriate workstations, were informed verbally about the nature of the 
experiment, and asked to read a set of instructions pertaining to both the experiment and 
the ELICIT environment. Subjects had 10 minutes to read the instruction set quietly. 
Subjects were encouraged to ask questions about the functional nature of the 


experimental settings throughout this process. Once subjects read the instructions they 
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were offered a ten-minute break and were encouraged to discuss their approach to the 
scenario. At the end of the break, the students took their places at the workstations and 


the moderator began the simulation. 


1. Hierarchy 


The hierarchy group is stratified into three functional tiers. The Senior Supervisor 
(Alex, labeled “1’’) may share factoids with Middle Managers (i.e., labeled “2,” “6,” 
“10,” “14”) at any time. Middle Managers may share factoids with the Senior Supervisor, 
other Middle Managers or their subordinate team Operators (labeled “3,” “4,” “S” in the 
“2” group and so forth). Operators may share factoids with other operators in their team 
and with the Middle Managers. No one may post globally (which would share factoids 
with ALL individuals), and each solution group in the hierarchy may only Pull factoids 
pertaining to their specific group’s task, who, what, where, and when. This manipulation 
administers the nature of the Hierarchy represented in Figure 2 and follows that of 


Leweling & Nissen, 2007. 


“A Senior Supervisor (i.e.) is responsible for the intelligence organization as a 
whole, and has four Middle Managers reporting directly. Each such leader in turn has 
three Operators reporting directly, and is responsible for one set of details associated with 
the terrorist plot. For instance, Middle Manager 2 and team would be responsible for the 
“who” details (e.g., which terrorist organization is involved) of the plot, Middle Manager 
6 and team would be responsible for the “what” details (e.g., what the likely target is), 
and so forth for “when” and “where.” Subjects are shown this organization chart, told of 
their responsibilities within the organization, and provided with a short description of the 
hierarchy. Additionally, the ELICIT software limits subjects’ Post and Pull access to 
specific common screens within this manipulation. Specifically, those players in the 
“who” group, for instance, are allowed to Post to and Pull from only one of the four 
common screens (i.e., the “who” screen) noted above. Comparable restrictions apply to 
players in the other three functional groups. The only exception applies to the Senior 


Supervisor 1, who has post-pull access to all four common screens.” 
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Who Group What Group Where Group When Group 


Figure 3. Hierarchy 


2. Edge Organization 


The Edge Organization is very different. At any time, any subject assigned to an 
Edge organization may: a) share factoids with any other member; b) post factoids to, c) 
pull factoids from any solution set, Who, What, Where and When; or d) Identify with 
partial or complete answers to who, what, where, or when in the terrorist plot. In the Edge 
organization manipulation, there are no pre-assigned leaders or functional groups 
established in advance of the experiment. As noted above, the players are pre-assigned to 
specific roles (i.e., pseudonyms) within the game, but the various roles reflect no 
hierarchical or functional differences from one another. Subjects are told about the 
functional design in an Edge organization and informed that they may self-organize as 
they see best to solve the test case. Figure 2 characterizes the nature of this Edge 


manipulation. 
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Figure 4. Edge Organization 


Without an overall leader or functional groups, subjects must decide for 
themselves who works on which aspects of the problem, and who posts, pulls and 
exchanges information with whom. Edge subjects are encouraged to discuss their 
approach to the problem during the ten-minute break. With the Edge manipulation, the 
ELICIT software does not limit subjects’ Post and Pull access to specific common 


screens, and any player can post to and pull from any of the four common screens (i.e., 


oe 99 66 99 66. 99 66. 


who,” “what,” “when,” “where’”). Consistent with the other manipulation is the Share 
function, through which any player can share factoids directly with any other. The game 
ends after 45 minutes of game play and all players identify the plot details. The 
moderator prompts all subjects to identify with their best assessment before the end of the 


simulation. 
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3. Trust 


The trust manipulation used in the experiment is independent of the organization 
manipulation summarized above. It is based on the three perceived factors of trust 
expressed in Mayer, Davis and Schoorman’s study. Trust as a treatment is communicated 
to the subjects through expressions that communicate high levels of benevolence, ability 


and integrity in the instructions to subjects. See Appendices A and B for details. 


Benevolence is demonstrated with the statement, “Members of your community 
share information freely with a general orientation toward doing good to others. We are 
impressed with this orientation and are encouraged by the positive interactions among 
your fellow cohort members.” Ability is applied to all subjects with the suggestion that, 
“your intellect, varying skills, and past experience lead us to believe that you are well 
qualified to solve the terrorist threat problem.” Integrity exists in the protocol and 
treatment by confirming that, “your actions will be consistent, congruent, and credible 


with established protocols and guidelines.” 


4. Mistrust 


The mistrust manipulation used in the experiment is independent of the 
organization manipulation summarized above also. It is based on undermining the three 
perceived factors of trust expressed in Mayer, Davis and Schoorman’s study, or showing 
an absence of these factors. Mistrust as a treatment is communicated to the subjects 
through expressions that communicate low levels of benevolence, ability and integrity in 
the instructions to subjects. See Appendices C and D for details. The protocol item that 
impedes benevolence is the expression, “Members of your community normally work 
well together, but frequently withhold information from each other. We are unsure about 
how you interact among your fellow cohort members and question whether negative 
interactions have affected your relationships” and “previous sessions reveal that some 
individuals take pride in undermining team cohesion and effectiveness by generating and 
releasing false information or by non-participation in the exercise.” Similarly, the 
subjects’ ability is put under suspicion by suggesting that “we have yet to assess your 


intellect and skills, and wonder whether past experience qualifies you to solve the 
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terrorist threat problem as a group.” Last, the protocol attacks the group and their 
individual integrity by announcing that “we are discouraged that when it comes to 
solving critical problems in group settings such as this that your actions may not be 
consistent, congruent, and credible with established protocols and guidelines. Simply put, 


be wary of moles and free-riders.” 


Research suggests that in organizations with high levels of perceived 
benevolence, ability and integrity, an environment of trust can and usually does develop. 


In the absence of these factors, mistrust fills the vacuum. 


The following section discusses data analysis and the results. It also mentions the 
correlation analysis conducted, the multivariate analysis done, the performance, 


interactive effects, and finally, trust conditions. 
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IV. ANALYSIS AND RESULTS 


This section reviews the results from the ELICIT simulation. We examine the 
performance data of individual players in the four conditions to determine 1) any 
correlations between performance parameters, 2) factors that may significantly affect 
performance among the various treatments, and 3) any comparisons of performance 
among the different treatments. This section begins with a summary of the descriptive 
statistics. The next segment examines the correlation between the dependent variables, 
(accuracy scores and speed scores). The next two segments examine a multivariate 
analysis followed by univariate analyses. The section then provides a summary of 
support for the research hypotheses. Finally, the section concludes with key findings. 
The results of our analysis offer insight into preferable combinations of organization 


types and trust conditions to optimize effectiveness. 
A. ANALYSIS AND DESCRIPTIVE STATISTICS 


The primary analysis consists of comparing the mean scores of the different 
treatments. The analysis is conducted using mean scores from the two measures of 
problem solving effectiveness (the dependent variables): individual accuracy scores and 
individual speed scores. Table 1 shows the descriptive statistics of individual 
performance, displaying the mean results and standard deviations within each group. The 
first four rows of the table correspond to the four treatment groups and reflect interaction 
effects between the manipulations of organization type and trust condition. The next two 
rows correspond to main effects of the organization manipulation, holding the trust 
condition constant. The last two rows correspond to main effects of the trust 


manipulation, holding the organization type constant. 
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Organization Type| Trust Condition | N | Accuracy Mean | Accuracy Std. Deviation | Speed Mean | Speed Deviation 
Edge High 0.05 0.25 0.03 








Edge Low : 0.04 





Hierarchy High : 0.05 





Hierarchy Low : 0.05 





Edge High + Low F 0.03 





Hierarchy High + Low 


Edge + Hierarchy 
Edge + Hierarchy 

















Table 1. | Descriptive Statistics 


B. CORRELATION BETWEEN DEPENDENT VARIABLES 


A correlation analysis is first conducted to determine the relationship between 
individual accuracy scores and individual speed scores. A positive correlation is found 
between the two scores, as illustrated in Table 2. This shows that individuals scored 
higher on accuracy if they answered earlier in the experiment, while accuracy scores 
decreased as the simulation approached the 45-minute termination point. This result 
seems counter intuitive because one would expect that more time spent working on a 
problem results in more accurate answers. The positive correlation is a result of the 
requirement that every participant had to provide an answer prior to the conclusion of 
each experiment. Consequently, approximately 78% of participants answered within the 
last 10 minutes of the experiment. The “late” answering participants scored a mean 
accuracy of .34 compared to the rest of the participants who answered earlier and scored 
a mean accuracy of .55. This result implies that the participants who answered during the 
last 10 minutes of the experiment probably did so at the urging of the administrators and 
not because they had correctly assembled a complete solution. Because the dependent 


variables are correlates of one another, a multivariate analysis is appropriate. 
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Performance Measure Speed 


Speed Pearson Correlation -281(**) 
Sig. (2-tailed) 


Accuracy Pearson Correlation 


Ns 





Table 2. Correlation Between Performance Measures 


Given that the simulation occurred over a two-day period, we test whether the 
mean scores from day | differed from those of day 2. We use an analysis of variance 
(ANOVA) to examine those differences. We find that in terms of accuracy, there is no 
significant difference. However, in terms of speed, we find that day 2 was faster with 
marginal significance at the .1 level, again suggesting that the administrators encouraged 


participants to identify and finish the simulation slightly more rapidly than on day 1. 
C. MULTIVARIATE ANALYSIS 


Next, we perform a multivariate analysis of variance (MANOVA). Table 3 shows 
the summary results. The main effect of our Organization Type manipulation is 
significant at the 0.05 level, and the main effect of our Trust manipulation is significant at 
the 0.1 level. The interaction effect of our combined Organization Type and Trust 
manipulation is highly significant. The interaction between organization type and trust 


appears to be powerful. 


Value Hypothesis df Error df Significance 
izati Pillia’s Trace 
Wilk’s Lambda 
Hotelling’s Trace 
Roy's Largest Root 


Trust Condition Pillia’s Trace 


Wilk's Lambda 
Hotelling’s Trace 
Roy's Largest Root 

Organization Type *  Pillia’s Trace 

Trust Condition Wilk's Lambda 
Hotelling’s Trace 
Roy's Largest Root 

Design: Intercept+Organization Type+Trust Condition + Organization Type * TrustType 





Table 3. Multivariate Results 
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D. UNIVARIATE ANALYSES 


We also examine how speed and accuracy vary separately across our 
manipulations through a series of Factorial ANOVA calculations. Table 4 shows the 
results of the ANOVA using speed scores as the dependent variable. The ANOVA 
reveals that taken independently the main effects, (organization type and trust condition), 
are not significant; however, the interaction between the two main effects is highly 
significant, (p<.001). 


Type Ill 
Sum of 
Squares 
586(a) 
0.022 
0 





2.397 
R Squared = .197 (Adjusted R Squared = .178) 
Dependent Variable: Speed 


Table 4. Univariate Results - Speed 


Figure 5 delineates the results of the mean speed scores. As summarized in the 
table above, for a given level of trust (i.e., trust or mistrust condition), speed performance 
across the organization manipulation does not appear to vary much. The same is apparent 
for speed across the trust manipulation for a given organization type (i.e., Hierarchy or 
Edge). However, notice that the lines in the figure cross, reflecting substantial variation 
when both organization type and trust condition are considered. This depicts the highly 
significant interaction effect noted above. When the level of trust is low, the Hierarchy 
outperforms the Edge in terms of speed. It appears as though the hierarchical organization 
structure enables participants to overcome the mistrust and work comparatively more 
quickly than in the Edge despite a low-trust environment. Alternatively, when the level of 


trust is high, the Edge outperforms the Hierarchy. It appears as though the Edge 
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organization structure enables participants to work much more quickly than in the 
Hierarchy when trust is high. Notice that the Edge organization in the trust condition 


produces the highest overall performance in terms of speed. 
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Figure 5. | Mean Speed Scores 


Table 5 shows results using accuracy scores as the dependent variable. Notice in 
this case that both main effects are significant at the 0.05 level, and the interaction effect 
is significant at the 0.01 level. Unlike the analysis above, in which neither main effect is 
significant in terms of speed as a performance measure, both the organization type and 
trust condition have strong influences on performance in terms of accuracy. Like the 


analysis above, the interaction of organization and trust is strong. 
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Type Ill 
Sum of 


7.161 





R Squared = .197 (Adjusted R Squared = .178) 


Table 5. | Univariate Results - Accuracy 


Figure 6 delineates the results of the mean accuracy scores. Their interpretation is 
a bit more complicated than that for speed outlined above. When the level of trust is low 
(i.e., the mistrust manipulation), there is negligible performance differential between 
organizational types in terms of accuracy. When mistrust pervades, the organization type 
does not appear to make much difference. Alternatively, when the level of trust is high, 
The Edge organization outperforms the Hierarchy. Interestingly, in the Hierarchy it does 
not appear to matter whether trust is present or not in terms of accuracy; performance is 
roughly the same across both trust and mistrust conditions. Notice too how the 
combination of Edge organization type and trust condition produces the highest overall 


performance in terms of accuracy. This parallels the result in terms of speed noted above. 
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Figure 6. | Mean Accuracy Scores 


EK. SUPPORT FOR HYPOTHESES 


Table 6 summarizes results in terms of the hypotheses. Each hypothesis is listed 
in the first column, and the corresponding statistical support in terms of multivariate and 


univariate analyses is noted across the other columns. 
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Statistical Support 
MANOVA ANOVA - speed ANOVA - accurac 


1. Edge Organization Type 
outperforms Hierarchy Organization : 
Type regan dless of aside sit dition. supported not supported supported (main effect) 
1(a). Edge high trust condition 
— rey ae ee supported supported {interaction effect) 
1(b). Edge mistrust condition 

not supported not supported {interaction effect) 


outperforms Hierarchy mistrust 
condition. 


2. Trust outperforms mistrust 

. ela ' : 
ee eee supported not supported supported (main effect) 
2(a). Trust in Edge Organization 
Type outperforms mistrust in Edge ‘said rted ae aad 
Organization Type. supporte supporte {interaction effect) 


2(b). Trust in Hierarchy Organization 
Type outperforms mistrust in 
Hierarchy Organization Type. 





not supported not supported {interaction effect) 


Table 6. | Hypothesis Statistical Support Summary 


We find that Hypothesis | is partially supported. The multivariate main effect for 
organization type is statistically significant, but univariate results are mixed: insignificant 
in terms of speed but significant in terms of accuracy. Hypothesis 1(a) suggests that the 
Edge Trust Condition outperforms the Hierarchy Trust Condition. The evidence as 
indicated by Tables 4 and 5 and Figures 2 and 3 supports this hypothesis. Hypothesis 1(b) 
suggests that the Edge Mistrust Condition outperforms the Hierarchy Mistrust Condition. 
The evidence as indicated by Table 4 and 5 and Figures 2 and 3 does not support this 
hypothesis. 


We find that Hypothesis 2 is also partially supported. The multivariate main effect 
for trust is statistically significant, but univariate results are mixed: insignificant in terms 
of speed but significant in terms of accuracy. Hypothesis 2(a) suggests that the Edge 
High Trust Condition will outperform the Edge Low Trust Condition. The evidence as 


indicated by Tables 4 and 5 and Figures 2 and 3 supports this hypothesis. Hypothesis 
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2(b) suggests that the Hierarchy High Trust Condition will outperform the Hierarchy Low 
Trust Condition. The evidence as indicated by Table 4 and 5 and Figures 2 and 3 does not 


support this hypothesis. 
F. KEY FINDINGS 


This project’s laboratory experiment and analysis consider two primary concepts: 


optimal organization type and optimal trust condition. 


The data provide mixed support for Hypothesis 1. We find that within a High 
Trust environment, the Edge Organization outperforms the Hierarchy Organization. 
However, in a Low Trust environment, no difference occurs in performance between the 
two organizations. The data also provide mixed support for Hypothesis 2. This finding 
intimates the unexpected result whereby trust conditions facilitate differences in 
performance only within an Edge Organization. Additionally, the highest levels of 
performance, in terms of both speed and accuracy, are observed in the Edge High Trust 


condition. 


These results can provide three important insights for organizational designers 
when considering accuracy as a measure of performance. First, the Edge Organization is 
very sensitive to trust, while the Hierarchy Organization is not. This results because a 
Hierarchy Organization operates on authority-driven decisions, rules, policy adherence, 
and skill sets assigned within a chain of command; members of an Edge Organization, on 
the other hand, accomplish tasks relying on one another’s skills and knowledge. These 
relationships are strengthened in environments with high levels of trust. Second, in a 
Trust Condition, the Edge Organization is the preferable structural design for optimal 
performance. Third, in the Hierarchy, one can expect similar performance in either the 


Trust or Mistrust Condition. 


Section V builds upon the results of this analysis to offer implications for practice 
and recommendations for leaders, managers, and organizational designers. The section 
also provides recommendations for further research in order to expand the body of 


knowledge pertaining to organizations and implications of trust. 
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V. DISCUSSION, RECOMMENDATIONS, AND CONCLUSIONS 


In this section, we discuss implications of our laboratory experiments studying the 
effects of trust and organizational type on problem solving effectiveness, and make 


recommendations for future research. 
A. DISCUSSION 


This section discusses the usefulness of the experimental results for both 
organizational design and trust conditions. Additionally, some recommendations for 


future research are made. 


This research is partly an attempt to illustrate better how Edge organizations can 
improve effectiveness in some military decision making environments. Edge is an 
appropriate structure for problem-solving in temporary teams. In the ELICIT electronic 
environment, Edge has the best communication capability for effectiveness. The Edge 
structure benefits from the decentralization of power and influence, minimal formality 
and omni-directional information flow. The results support the research hypothesis that in 
a High Trust condition, an Edge organization accesses faster and more accurate problem 
solving capability by its flexible and self-organizing, responsive structure. However, 
Edge organizations appear to be more sensitive to trust. An Edge structure in a mistrust 
condition does not appear able to exploit the very benefits of Edge. The free-form 
communication and synergistic cooperation that is typical and sought after appears to be 
undermined or disrupted in the absence of organizational trust in the form of perceived 
integrity, benevolence and competence. In a high-trust condition, Edge outperforms a 
hierarchy in speed and accuracy. The experiment confirms that in groups with high trust, 
the Edge organization will solve problems with better accuracy and speed than a mistrust 
condition. In general, problem-solving effectiveness is optimized in an Edge, Trust 


environment. 


Hierarchy is a more typical organization structure in a staff headquarters or 


operational command. Tiers of power and influence are often necessary boundaries to 
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allow managers to focus on one, linear operational task at a time. In general, Hierarchy is 
not as sensitive to trust and mistrust. An organization’s problem-solving effectiveness in 
a Hierarchy would be much the same whether the organization has developed trust or not. 
This is actually a great safety mechanism in the Hierarchy as expressed in the operational 
military. Most missions do not require or allow for a blossoming of organizational trust 
from the Commanding Officer down to the newest enlistee out of Basic Training. In a 
Hierarchy, effectiveness may not explicitly require organizational trust, in the form of 
perceived integrity, benevolence and competence, but trust is still a goal worthy of 
development through training and culture-management in the military. The apparent 
benefit of a Hierarchy is its inherent and necessary compliance to the organizational 
structure and roles in spite of varying individual perceptions of trust. In a mistrust 
environment, a Hierarchy is best suited to expedite complex problem solving and produce 


precise results. 


B. IMPLICATIONS 


To address the most simple and direct interpretation of the research and the 
supporting literature best, the following acts as a guide to organization type and trust 


conditions. 


In designing or restructuring an organization when the luxury of being proactive 
exists, and outputs are not urgent: 


e Commit to build trust throughout the organization, and then form Edge 
teams to solve specific problems. 


or 


e Build Edge designed, cross-functional teams, and then develop trust in the 
team members before assigning outputs. 


In designing or restructuring an organization (re-active, crisis, urgent): 


° If trust is absent or suspect, or the trust condition is unknown, and it is 
necessary to expedite the process, select a Hierarchy organization with 
clear roles, responsibilities, and explicitly define desired outcomes. 


e If high-trust is present and it is essential to must expedite the process, then 
form Edge teams to solve specific problems. 
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C. RECOMMENDATIONS FOR FUTURE RESEARCH 


Conclusions and recommendations from this research are based on a substantial 
amount of evidence to support the impact of organization type and trust conditions on 
organizational effectiveness. The experimental design creates an easy set of experimental 
outputs for analysis, and the evidence supports most of the hypotheses. However, there is 
adequate room for improvement in the execution of the design, and immense value in 


further research. The recommendations for further research include the following. 


Conduct the same basic experimental design, but provide more explicit 
instructions before the ELICIT session begins, or allow a short “orientation” to the 
ELICIT game. Ensure that moderators explain the nature of ELICIT communication 
protocol clearly and well. In particular, one could modify the experiment protocol to 
ensure that all subjects know how to Identify all specific aspects of the ELICIT. 

Example: 
“What” complete and correct answer: The Southern Oil Pipeline Terminal 
“What” answer given in Identify: Pipeline 


“Where” complete and correct answer: Psiland 


“Where” answer given in Identify: Terminal (a “What” answer submitted as a 


“Where’’) 
“Who” complete and correct answer: The Violet Group 
“Who” answer given in Identify: The Lion 
“When” complete and correct answer: March 15" at 1100 am 
“When” answer given in Identify: The 15" at eleven 


Such refinement in protocol will likely result in better accuracy scores from future 


subjects. 


Conduct the same basic experimental design, but exercise more strict and 


consistent controls during the subject interaction. In particular, one could modify the 
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experiment protocol to ensure that all subjects recognize that the mistrust condition 
means that every factoid Shared and Posted is subject to suspicion. Simply, future 


experimenters must create conditions of mistrust that alter behavior in all subjects. 


Conduct the same basic experimental design, but lengthen the ELICIT sessions. 
This will enable a more gradual adjustment to the experimental methods of 
communication and problem solving. Even though the complete set of factoids is 
distributed within the first 20 minutes, there are many players who seem to struggle not 
with the information itself, but the mechanics of Share, Post, and Pull. In particular, one 
could modify the experiment protocol to ensure that the ELICIT game is played for not 
less than 90 minutes or until all subjects have Identified with confidence in their answer, 
and without being prompted to do so by the moderators. This will also build more 
significance in the relationship between accuracy and speed, as the final Identify for each 
individual will be the best estimate in the time that actually elapsed rather than measuring 


the subject’s best estimate with a prompted Identify constraint. 


Conduct the same basic experimental design, but only accept time/speed as a 
measure of effectiveness for accurate and complete and answers. Certainly, in a problem 
solving exercise where people’s lives are at risk, there should be an expressed reluctance 
by subjects to submit hasty and reckless solutions early in the process rather than accurate 
and complete answers later. Additionally, in the real world of counterterrorism, only an 
answer that is accurate and complete would suffice to allow the prevention of a terrorist 


plot from occurring. 


Modify the experimental design to examine the different components of trust— 
perceived integrity, benevolence and competence—independently as well as in 
combination. The current design examines all three components as a single manipulation, 
but the comparative effects of each component may differ. Further, trust can be viewed as 
a dependent variable as well as an independent one. An additional experimental design 
could examine the effect of different organizational designs, leadership styles, 
communication protocols and other factors on the emergence and development of trust 


over time (e.g., with pretest-posttest measures). 
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In summary, this study of trust and its effect on organizations concludes that high 
trust is the preferred condition when Edge is selected over Hierarchy. It is also 
discovered that Edge Organizations within a high trust condition outperform other 
organization and trust combinations when solving complex problems in rapidly 


established groups. 


Factors of perceived trustworthiness in this research include ability, benevolence 
and integrity. In a military or civilian environment, trust is paramount to maximizing 
effectiveness. In concert with numerous studies in trust and organizational design, it is 
found that these factors are necessary personality traits for leaders to seek out and to 
develop in an effort to establish and maintain trust. Studies and the results indicate that 
other organizational dynamics exist such as the interaction between trust conditions and 
organizational design, leadership style, and motivational structure. This study of trust in 
differing organizational groups adds to the credible and abundant research on trust and 


organizational design and offers unique potential for further experimentation. 
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APPENDIX A. 


Instructions to "A'' Community (Edge Trust, not shown to subjects) 


You have been randomly assigned to the "A" community group. Your community is a 
group of intelligence officers whose task is to reveal the identity of a threatening terrorist 
group, plot, location, and date/time of an impending attack. Working with the members 
for your community, you will share information and piece the data together to provide a 
solution. 


Members of your community share information freely with a general orientation toward 
doing good to others. We are impressed with this orientation and are encouraged by the 
positive interactions among your fellow cohort members. Moreover, your intellect, 
varying skills, and past experience lead us to believe that you are well qualified to solve 
the terrorist threat problem. We have every reason to believe that when it comes to 
solving critical problems in group settings such as this that your actions will be 
consistent, congruent, and credible with established protocols and guidelines. 


Group communications capabilities are available to enable you to share information with 
any member of your group. As a member of “A” community, your knowledge and 
unique experiences are valuable assets, and you are expected to give best effort to 
the success of your team. You are invited to establish communication with any and 
all members of your group and organize your group in any way that best suits your 
problem solving. 


Software 


The software supports two ways of informing Operators about factoids you have 
“discovered.” You can Share a factoid directly with another subject using the share tab 
or you can Post a factoid to one of four websites. There are separate Who, What, Where 
and When sites. Subjects can post intelligence information [factoids] they are aware of to 
a website that others in their group can simultaneously see and update. Though these 
areas are called websites, the information display is provided by the experiment software 
and not by the Internet. 


e Factoids in your inbox can be copied into your MyFactoids list by 
selecting the factoid and clicking on the Add to MyF actoids action. 


e POST: To Post a factoid to a website that can be viewed by your 
Operators, select the factoid from your inbox or MyFactoids list that you 
wish to post, click on the Post action, and select the website you wish to 
post to. 
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e SHARE: To Share, select the factoid from either your inbox or your 
MyFactoids list that you wish to share, click on the Share action, and 
select the anonymized name of the person with whom you want to share. 
This sends the factoid to the selected subject’s inbox message list. 


e The Add to My Factoids action can also be used to copy a factoid from a 
website to your MyFactoids list. To copy a factoid from a website to your 
MyFactoids list, select the factoid you wish to copy from a website and 
click on the Add to MyFactoids action. 


In summary, a subject can Add an item in their inbox to their MyFactoids list or Share or 
Post it directly with another member of their group. Items in their MyFactoids list can 
either be Shared or Posted to a website. An item on a website can be Added to a subject's 
MyFactoids list. 


"A" members also have shared information tools. 


e To get a summary list of all the factoids in your MyFactoids list, click on 
the MyFactoids tab in the middle of your screen. 


e To find out your role information and how other members of your group 
see you, click on the “How I’m seen” tab. 


e To get a list of all the members in your group, with information about their 
role and country, click on the “What I see” tab. 


° To access information from your team website, click on the website that 
you wish to view. To update the website with the latest information that 
has been posted to it, click on the Refresh action a the top of the screen, 
while viewing the website. 


During each experiment round, you are free to work on any aspect of the task. The 
experiment software provides communications tools that you can use to interact with the 
other members of your group and communicate information. 


When you think that you have identified the who, what, where and when of the adversary 
attack, click on the Identify tab at the top of your screen and enter free text messages that 
identify the who, what, where and when of an adversary attack. Partial answers are 
accepted. You may Identify more than once. 


e The who is a group (for example the blue group). 
e The what is a type of target (for example an embassy or religious school or 
dignitary). 
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e The where is the country in which the attack will take place (for example 
Alphaland). 


e The when is the month, day and hour on which the attack will occur (for 
example December 15, at 3:00 am.) 


You can re-refer to these instructions during the course of the experiment by clicking 
again on the URL in the moderator message. 


When you have finished reading this important group background information, and are 
ready to begin the practice round, click the Ready button in the upper left corner of your 
screen. 


You may have the next 10 minutes to re-read these instructions, take a break or 
share recommendations with your team mates. 
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APPENDIX B. 


"B" Community (Hierarchy Trust, not shown to subjects) 


You have been randomly assigned to the "B" organizational group. Your community is a 
group of intelligence officers whose task is to reveal the identity of a threatening terrorist 
group, plot, location, and date/time of an impending attack. Working with the members 
for your community, you will share information and piece the data together to provide a 
solution. 


Members of your community share information freely with a general orientation 
toward doing good to others. We are impressed with this orientation and are 
encouraged by the positive interactions among your fellow cohort members. 
Moreover, your intellect, varying skills, and past experience lead us to believe that 
you are well qualified to solve the terrorist threat problem. We have every reason to 
believe that when it comes to solving critical problems in group settings such as this 
that your actions will be consistent, congruent, and credible with established 
protocols and guidelines. 


In the "B" organization, there are four teams of four members each plus an overall cross- 
team coordinator. The four teams are organized along the lines of a traditional 
hierarchical command structure, each with a leader. This diagram shows the hierarchical 
relationship between a Senior Supervisor (E5), the four Middle Managers (A4, B4, C4 
and D4) and their Operators, 1-3: 





A: WHO «Bs WHAT - C: WHERE _D: WHEN 


Team A is focused on Who, team B on What, team C on Where and team D on When. 
The overall coordinator coordinates information between the team leaders across team 
boundaries. Note that the above diagram is an organization chart and not a 
communications chart. 
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Software 


The software supports one way of informing Operators about factoids you have 
“discovered.” You can Share a factoid directly with another subject using the Share 
tab. The following restrictions apply: 


Senior Supervisors can “Share down” with Middle Managers. 
Operators can “Share across” with other Operators. 

Operators can “Share up” with Middle Managers. 

Middle Managers can “Share up” with the Senior Supervisor. 


Each of the 4 teams in your group has its own website. Subjects can post intelligence 
information (factoids) they are aware of to a website that others in their group can 
simultaneously see and post to. Though these areas are called websites, the information 
display is provided by the experiment software and not by the Internet. In the "B" 
organization each member of a team can see and update their team’s website. For 
example, all members of the “where” team can see items posted to their team’s “where” 
website. The cross-team coordinator can see all four of the typed websites 


e Factoids in your inbox can be copied into your MyFactoids list by 
selecting the factoid and clicking on the Add to MyFactoids action. 


e To Post a factoid to a website that can be viewed by your Operators, select 
the factoid from your inbox or MyFactoids list that you wish to post, click 
on the Post action, and select the website you wish to post to. 


e To Share, select the factoid from either your inbox or your MyFactoids list 
that you wish to share, click on the Share action, and select the 
anonymized name of the person with whom you want to share. This sends 
the factoid to the selected subject’s inbox message list. 


e The Add to My Factoids action can also be used to copy a factoid from a 
website to your MyFactoids list. To copy a factoid from a website to your 
MyFactoids list, select the factoid you wish to copy from a website and 
click on the Add to MyFactoids action. 


In summary: a subject can Add an item in their inbox to their MyFactoids list or Share or 
Post it directly with another member of their group. Items in their MyFactoids list can 
either be Shared or Posted to a website. An item on a website can be Added to a subject's 
MyFactoids list. 
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"A" members also have shared information tools. 


e To get a summary list of all the factoids in your MyFactoids list, click on 
the MyFactoids tab in the middle of your screen. 


e To find out your role information and how other members of your group 
see you, click on the “How I’m seen” tab. 


e To get a list of all the members in your group, with information about their 
role and country, click on the “What I see” tab. 


e To access information from your team website, click on the website that 
you wish to view. To update the website with the latest information that 
has been posted to it, click on the Refresh action a the top of the screen, 
while viewing the website. 


During each experiment round, you are free to work on any aspect of the task. The 
experiment software provides communications tools that you can use to interact with the 
other members of your group and communicate information. 


When you think that you have identified the who, what, where and when of the adversary 
attack, click on the Identify tab at the top of your screen and enter free text messages that 
identify the who, what, where and when of an adversary attack. Partial answers are 
accepted. You may Identify more than once. 


e The who is a group (for example the blue group). 

e The what is a type of target (for example an embassy or religious school or 
dignitary). 

e The where is the country in which the attack will take place (for example 
Alphaland). 

e The when is the month, day and hour on which the attack will occur (for 


example December 15, at 3:00 am). 


You can re-refer to these instructions during the course of the experiment by clicking 
again on the URL in the moderator message. 


When you have finished reading this important group background information, and are 
ready to begin the practice round, click the Ready button in the upper left corner of your 
screen. 


You may have the next 10 minutes to re-read these instructions, take a break or 
share recommendations with your team mates. 


47 


THIS PAGE INTENTIONALLY LEFT BLANK 


48 


APPENDIX C. 


"A" Community (Edge Mistrust, not shown to subjects) 


You have been randomly assigned to the "A" community group. Your community is a 
group of intelligence officers whose task is to reveal the identity of a threatening terrorist 
group, plot, location, and date/time of an impending attack. Working with the members 
for your community, you will share information and piece the data together to provide a 
solution. 


Members of your community normally work well together, but frequently withhold 
information from each other. We are unsure about how you interact among your 
fellow cohort members and question whether negative interactions have affected 
your relationships. Moreover, we have yet to assess your intellect and skills, and 
wonder whether past experience qualifies you to solve the terrorist threat problem 
as a group. Previous sessions reveal that some individuals take pride in undermining 
team cohesion and effectiveness by generating and releasing false information or by 
non-participation in the exercise. We are discouraged that when it comes to solving 
critical problems in group settings such as this that your actions may not be 
consistent, congruent, and credible with established protocols and guidelines. 
Simply put, be wary of moles and free-riders. 


Software 


The software supports two ways of informing Operators about factoids you have 
“discovered.” You can Share a factoid directly with another subject using the share tab 
or you can Post a factoid to one of four websites. There are separate Who, What, Where 
and When sites. Subjects can post intelligence information [factoids] they are aware of to 
a website that others in their group can simultaneously see and update. Though these 
areas are called websites, the information display is provided by the experiment software 
and not by the Internet. 


e Factoids in your inbox can be copied into your MyFactoids list by 
selecting the factoid and clicking on the Add to MyFactoids action. 


e POST: To Post a factoid to a website that can be viewed by your 
Operators, select the factoid from your inbox or MyFactoids list that you 
wish to post, click on the Post action, and select the website you wish to 
post to. 


e SHARE: To Share, select the factoid from either your inbox or your 
MyFactoids list that you wish to share, click on the Share action, and 
select the anonymized name of the person with whom you want to share. 
This sends the factoid to the selected subject’s inbox message list. 
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e The Add to My Factoids action can also be used to copy a factoid from a 
website to your MyFactoids list. To copy a factoid from a website to your 
MyFactoids list, select the factoid you wish to copy from a website and 
click on the Add to MyFactoids action. 


In summary, a subject can Add an item in their inbox to their MyFactoids list or Share or 
Post it directly with another member of their group. Items in their MyFactoids list can 
either be Shared or Posted to a website. An item on a website can be Added to a subject's 
MyFactoids list. 


"A" members also have shared information tools. 


e To get a summary list of all the factoids in your MyFactoids list, click on 
the MyFactoids tab in the middle of your screen. 


® To find out your role information and how other members of your group 
see you, click on the “How I’m seen” tab. 


e To get a list of all the members in your group, with information about their 
role and country, click on the “What I see” tab. 


e To access information from your team website, click on the website that 
you wish to view. To update the website with the latest information that 
has been posted to it, click on the Refresh action a the top of the screen, 
while viewing the website. 


During each experiment round, you are free to work on any aspect of the task. The 
experiment software provides communications tools that you can use to interact with the 
other members of your group and communicate information. 


When you think that you have identified the who, what, where and when of the adversary 
attack, click on the Identify tab at the top of your screen and enter free text messages that 
identify the who, what, where and when of an adversary attack. Partial answers are 
accepted. You may Identify more than once. 


e The who is a group (for example the blue group). 

° The what is a type of target (for example an embassy or religious school or 
dignitary). 

e The where is the country in which the attack will take place (for example 
Alphaland). 

e The when is the month, day and hour on which the attack will occur (for 


example December 15, at 3:00 am.) 


You can re-refer to these instructions during the course of the experiment by clicking 
again on the URL in the moderator message. 


50 


When you have finished reading this important group background information, and are 
ready to begin the practice round, click the Ready button in the upper left corner of your 
screen. 


You may have the next 10 minutes to re-read these instructions, take a break or 
share recommendations with your team mates. 


51 


THIS PAGE INTENTIONALLY LEFT BLANK 


52 


APPENDIX D. 


"B" Community (Hierarchy Mistrust, not shown to subjects) 


You have been randomly assigned to the "B" organizational group. Your community is 
a group of intelligence officers whose task is to reveal the identity of a threatening 
terrorist group, plot, location, and date/time of an impending attack. Working with the 
members for your community, you will share information and piece the data together to 
provide a solution. 


Members of your community normally work well together, but frequently withhold 
information from each other. We are unsure about how you interact among your 
fellow cohort members and question whether negative interactions have affected 
your relationships. Moreover, we have yet to assess your intellect and skills, and 
wonder whether past experience qualifies you to solve the terrorist threat problem 
as a group. Previous sessions reveal that some individuals take pride in undermining 
team cohesion and effectiveness by generating and releasing false information or by 
non-participation in the exercise. We are discouraged that when it comes to solving 
critical problems in group settings such as this that your actions may not be 
consistent, congruent, and credible with established protocols and guidelines. 
Simply put, be wary of moles and free-riders. 


In the "B" organization, there are four teams of four members each plus an overall cross- 
team coordinator. The four teams are organized along the lines of a traditional 
hierarchical command structure, each with a leader. This diagram shows the hierarchical 
relationship between a Senior Supervisor (E5), the four Middle Managers (A4, B4, C4 
and D4) and their Operators, 1-3: 
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Team A is focused on Who, team B on What, team C on Where and team D on When. 
The overall coordinator coordinates information between the team leaders across team 
boundaries. Note that the above diagram is an organization chart and not a 
communications chart. 


Software 


The software supports one way of informing Operators about factoids you have 
“discovered.” You can Share a factoid directly with another subject using the Share 
tab. The following restrictions apply: 


Senior Supervisors can “Share down” with Middle Managers. 
Operators can “Share across” with other Operators. 

Operators can “Share up” with Middle Managers. 

Middle Managers can “Share up” with the Senior Supervisor. 


Each of the 4 teams in your group has its own website. Subjects can post intelligence 
information (factoids) they are aware of to a website that others in their group can 
simultaneously see and post to. Though these areas are called websites, the information 
display is provided by the experiment software and not by the Internet. In the "B" 
organization each member of a team can see and update their team’s website. For 
example, all members of the “where” team can see items posted to their team’s “where” 
website. The cross-team coordinator can see all four of the typed websites 


e Factoids in your inbox can be copied into your MyFactoids list by 
selecting the factoid and clicking on the Add to MyFactoids action. 


e To Post a factoid to a website that can be viewed by your Operators, select 
the factoid from your inbox or MyFactoids list that you wish to post, click 
on the Post action, and select the website you wish to post to. 


e To Share, select the factoid from either your inbox or your MyFactoids list 
that you wish to share, click on the Share action, and select the 
anonymized name of the person with whom you want to share. This sends 
the factoid to the selected subject’s inbox message list. 


e The Add to My Factoids action can also be used to copy a factoid from a 
website to your MyFactoids list. To copy a factoid from a website to your 
MyFactoids list, select the factoid you wish to copy from a website and 
click on the Add to MyFactoids action. 
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In summary: a subject can Add an item in their inbox to their MyFactoids list or Share or 
Post it directly with another member of their group. Items in their MyFactoids list can 
either be Shared or Posted to a website. An item on a website can be Added to a subject's 
MyFactoids list. 


"A" members also have shared information tools. 


e To get a summary list of all the factoids in your MyFactoids list, click on 
the MyFactoids tab in the middle of your screen. 


e To find out your role information and how other members of your group 
see you, click on the “How I’m seen” tab. 


e To get a list of all the members in your group, with information about their 
role and country, click on the “What I see” tab. 


e To access information from your team website, click on the website that 
you wish to view. To update the website with the latest information that 
has been posted to it, click on the Refresh action a the top of the screen, 
while viewing the website. 


During each experiment round, you are free to work on any aspect of the task. The 
experiment software provides communications tools that you can use to interact with the 
other members of your group and communicate information. 


When you think that you have identified the who, what, where and when of the adversary 
attack, click on the Identify tab at the top of your screen and enter free text messages that 
identify the who, what, where and when of an adversary attack. Partial answers are 
accepted. You may Identify more than once. 


e The who is a group (for example the blue group). 

e The what is a type of target (for example an embassy or religious school or 
dignitary). 

e The where is the country in which the attack will take place (for example 
Alphaland). 

e The when is the month, day and hour on which the attack will occur (for 


example December 15, at 3:00 am). 


You can re-refer to these instructions during the course of the experiment by clicking 
again on the URL in the moderator message. 


When you have finished reading this important group background information, and are 
ready to begin the practice round, click the Ready button in the upper left corner of your 
screen. 


35 


You may have the next 10 minutes to re-read these instructions, take a break or 
share recommendations with your team mates. 


Once the moderator is ready to commence ELICIT game play and subjects are 
appropriately assigned logged in, subjects indicate via the ELICIT client that they are 
ready to begin. Subjects are instructed not to reveal their pseudonyms to one another 
during the game. Indeed, they are instructed not to talk or communicate with one another 
during the game via any mechanism outside of the two summarized above (i.e., post-pull, 
share). 

Subjects are given incentives to play the game well, as participation and 
performance are actual class-time requirements, and graded. This is intended to be an 
incentive to ensure subjects cooperate and perform at their best. Each subject is instructed 
to use the Identify function as frequently as desired during game play, but each player in 
the game is expected to wait until he or she is relatively confident about the plot before 
publishing their best solution via the Identify command. The moderator prompts all 
subjects to Identify with their best answer before the game ends by announcing time 
milestones at the 35, 40, and 44 minute mark. The game ends after 45 minutes. Subjects 
are not told the results and the correct answers to the terrorist plot. “This represents in 
part the kind of equivocally inherent in intelligence work: analysts are rarely certain 
about any suspected plot, and many are required to work on multiple plots either 
simultaneously or sequentially. Again, we go to considerable lengths to enhance the 
realism of the game—and hence external validity of the results’ (Leweling & Nissen, 
2007). 


Controls 


We have elected to address the research question with a randomized two-factor 
quasi-experimental design. As noted above, the senior ranking person in each group is 
assigned as “Alex,” and is intended to play such specific role through the game. The role 
of leadership is only functionally significant in the hierarchy organizational structure 
because of the nature of linear communication flow. In the edge organization, Alex was 
not given any specific role definition by the moderator, and therefore did not exert any 
more responsibility than as a spokes-person accurately representing Edge and self- 
organizing team dynamics. Each version of the game is structurally equivalent, and both 
the ELICIT software and physical laboratory environments are consistent across the eight 
experimental sessions. To the extent possible, we distributed subjects into near-identical 
groups for equal distribution of gender, Service, military rank, and age, achieving the 
greatest uniformity between all eight groups. 


Treatment of Data 


The results of the ELICIT study will provide us with one data set per group. Each 
individual subject will be assigned two measures of effectiveness. 
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Accuracy rate of ELICIT solution: 


a. 


Each individual will receive a score of 1.0 for each correct aspect 
of the ELICIT solution derived from the Factoids (Who, What, 
Where, When Month, When Day, and When Time). 

The subject-group’s accuracy score is the mean of the 17 
participants’ accuracy scores. 

Calculations: 

((Who + What + Where) + (When Month + When Day + When 
Time)/3) /4 


((1.0 + 1.0 + 1.0) + (0.0 + 1.0 + 0.0)/3) /4 = 0.83 


Time in elapsed seconds submission of final ELICIT solution: 


a. 


Each individual will have elapsed time recorded upon their final 
Identify of their ELICIT solution derived from the Factoids. 
Baseline Time is established by the maximum time elapsed until 
the last individual submission of their ELICIT solution (2872 
seconds) 

The subject-group’s accuracy score is the mean of the 17 
participants’ accuracy scores. 

Calculations: 


(Baseline — Individual Identify Time) / Baseline 
(2872 — 2385) / 2872 = 0.1695 


ay, 
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